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Context 


* Unreal Engine 45 primary anti-aliasing solution 
— Referred to as Temporal AA in the engine 


e First used in the UEA Infiltrator tech demo 
e Several major revisions since then 
e Still ongoing work 
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* Deferred shading 
e Physically based 
e HDR 
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UE4 renderer 


Problem 


e Horrifically aliased input 

* Both geometric and shading aliasing 
* Mostly from subpixel features 

* Want temporal stability 
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MSAA? 


* Too expensive with deferred 
— Don't want to shade more than once per pixel 
・ Doesn't affect shading aliasing 
— More significant aliasing inside triangles than at their edges 
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Spatial filter” 


e МАД, FXAA, SMAA, etc. 

* Essentially edge finding, reduces stair stepping 
— Primarily not a stair stepping problem 

* No knowledge of subpixel features 


* Not temporally stable 
— Even on simple stair stepping 
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opecular Lobe filtering? 


e Toksvig, LEAN, УМЕ etc. 
* Filters shading input to prevent subpixel shading output 
* Difficult to pre-filter everything 


— Geometric features are major contributor 
— Often no existing unique roughness map 
— Procedural texturing 
— Still aliases 

* Screen space filter aliases 


— Misses subpixel features 
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Temporal filtering 


e Distribute samples over multiple frames 


e [ve had great success with this in the past 
- SSAO 
— SSR 
e Replaced spatial filter 
— Higher quality 
— Cheaper 
* Do the same with supersampling? 
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step 1: Static scene 
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Jittering 


e Adjust projection matrix 


ProjMatrix[2][0] += ( SampleX * 2.0f - 1.0f ) / ViewRect.Width(); 
ProjMatrix[2][1] += ( SampleY * 2.0f - 1.0f ) / ViewRect.Height(); 


Regular grid 
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sample pattern 


* Want a low discrepancy progressive sequence 


- No clustering in either space or time 


* Halton (2,3) worked well enough 
— Better than any HW MSAA sample ordering 


Halton 
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Moving average 


* Simple moving average 
— Not enough samples 
- п = 2 practical for color 


5р = 


le 


аре 


- n = 5 if luma only 


e Exponential moving average 
Nearly infinite number of samples with fixed storage 
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Exponential smoothing 


* When a is small exponential = simple 
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When to average”? 


* Before tone mapping 
— The physically correct location 
— Bright values dominate 
— Aliases badly with limited + of samples 


・ After tone mapping 


— All post filters flicker 
— Aliased input — aliased output 
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Tone map 


otraightforward tone map solution 


* Hybrid of before and after 
— Apply before all post 
— Tone map input 
— Accumulate samples 
— Heverse tone map output 


» Same AA quality as after tone mapping 
* Provides AAed input to post processing chain 


— No more flickering bloom 
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Better tone map solution 


* Tone mapping desaturates bright pixels 


* Weight samples instead based on luminance weight = 


— Maintains chroma 1 + luma 
- Perceptually closer to ground truth 

* No need to store the weight color 
i | ШЕШ ОГ) = 一 一 一 
- Rederive weight 1+luma 

— Saves GPRs нм color 
ә : 10°) —  —_—— 
See my blog post: [Karis1 3] с 
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Reconstruction filter 
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Box filter is not stable under motion 
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Gaussian fit to Blackman-Harris 3.3 
Support is “2 pixels wide 
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step 2: Dynamic scene 
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Reprojection 


* History for current pixel may be elsewhere on screen 
— May not exist at all 


* Use same velocity buffer calculation as motion blur 
o Remember to remove jitter 


UNREAL 


ENGINE 


Velocity accuracy 


* Need velocity (motion vectors) for everything 
— Motion without correct velocity will smear 
» Accuracy is super important 
— Minor imprecision will streak a static image 
— 16:16 RG velocity buffer 
e Can be tricky 
— Procedural animation 
— Scrolling textures 
— Almost opaque translucent objects 
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Motion on edges 


e Moving silhouette edges lose АА 

— Smooth AAed edge doesn't move with object 

— Effectively an aliased mask in the velocity buffer 
e Dilate velocity 

— Take front most velocity 
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_ Ghosting 


Ghosting 


* Depth compare? 

— All samples dont share same depth 
* Velocity weighting? 

— Shading changes 

— Iranslucency 
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Neighborhood clamping 


* New kid in town! 
— [Lottes11] , [Malan12] 

* Hestrict history to the range of current frame's local neighborhood 
— Assumes AA result is blend of neighbors 
— Clamp with min/max of 3x3 neighborhood 
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shaped neighborhood clamp 


e Simple clamp to min/max of 8 neighbors results in 3x3 box artifacts 


e Want min/max to appear filtered 
— Round out the shape 


e Solution: average 2 neighborhood's min/max 
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YCoCg box 


* Basic min/max is an AABB in RGB space 

* Ideally use convex hull of neighborhood colors 
— Too expensive 

e Orient box in luma direction 


— Luma has high local contrast 
— Chroma typically doesn't 
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Clip instead of clamp 


・ Constrain to a blend of history and neighborhood average 
e Clip line segment to box 
e Colors dont collect in box corners like clamping does 
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Translucency 


e Translucency is a poor fit for temporal 
— Single history 
— Single velocity 

* Ideally render translucency separate and composite 
— Cant unjitter depth buffer to compare against 

* Possible solution: 4xMSAA depth prepass 


— Alternate which sample to shade 
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Our translucency solution 


* “Responsive AA’ material flag 
・ Sets stencil when rendering translucency 


° Temporal AA pass tests stencil and uses minimal feedback 
— Unfortunately need >О feedback to prevent visible jittering 


* Only useful for small particles like sparks (DW о 
— Neighborhood clamping handles the rest {search Н o - 
Use Material Attributes E 
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4 Translucency 


Enable Separate Translucency 
E Enable Responsive AA vE 
Translucency Lighting Mode TLM Volumet w 


Translucency Directional Lightin [10 | 


Disable Depth Test O 
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Without Responsive AA 


Temporal АА 16 a firewall 


Visibility samples Spatial filters 


={Bicem) (DOF) Post) 


Temporal AA 
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Temporal АА is a firewall 


Visibility samples Spatial filters 
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Temporal AA 
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Flickering 


* Camera is static but some pixels flicker 
* Missing subpixel feature's history gets clamped 


— Often vertical or horizontal lines due to coherent jitter 
・ Clamping is an instantaneous impulse 
e This leads to saw tooth waves which appear as flickering 
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Bright edge missing in one frame 


UNREAL 


ENGINE 


Basic anti-flickering idea 


* Bias towards impulse frame 


— Shrinks amplitude of wave 


» Reduce exponential smoothing blend factor 


— Reduces recovery from impulses 


e Only where needed 


— Overly blurry results if done everywhere 


a a ли 
Bright edge missing in one frame 
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First anti-flickering attempt 


e Store historical variance data in alpha channel 
— Remember clamping events 
— Reduce blend factor and recover over time 
e Responsiveness issues 
— Can result in ghosting or blurring 
— Can bias towards aliased result 


UNREAL 


ENGINE 


Our current anti-flickering solution 


» Reduce blend factor when history is near clamping 
— Will happen after clamp events 
— Memory specific to event 
— Doesn't require additional storage 


* Not completely solved 


— Extremely difficult! 
— Impossible to solve multiple opposing clamps 
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Blurring: filter kernel 


* Mipmap bias all textures 
— Incorrect derivatives for supersampling 

* If low contrast then reduce filter kernel size 
— Technically aliases but looks fine 

* Gan add additional post sharpen filter 
— Mitchell 4.0 filter's negative lobes are >1 pixel away 
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Blurring: reprojection diffusion 


e Could use back and forth error compensation 
— Haven't had good results 

* Could store history at higher resolution 
— Really expensive 


* When reprojecting outside pixel reduce filter size and feedback 
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e Not its origir jl р 
- Really nice side effe 


— Doesnt cost anythi 8 


- Almost perfect m 


irror reflectio 


Мапу more potential applications 


e Stochastic transparency 
e Single sample anisotropic specular IBL 
e Soft shadows | 


・ Reduced steps for ray casting 
- Parallax occlusion mapping 


— Volumetric lighting 
・ Path tracing? 


e VH? 
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Future directions 


* Combine spatial and temporal 
* Separate translucency 
— Visibility and shading sample disconnect 
* Different jitter per pixel 
— Custom MSAA sample placement 
* More complete motion vectors 


ー Iranslucency 
ー Motion estimation 
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Conclusion 


* Temporal supersampling is production ready 
— High quality 
— High performance 


* Needs a lot of perceptual tuning 
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Thanks 


e Timothy Lottes, co-inventor 
e Epic 


— Rendering team 
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Full source code available! 


unrealengine.com 


$19/ mo + 5% 


Epic is hiring! 
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